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"Abstract 

Plan graphs are commonly used in planning to help compute 
heuristic “distance” estimates between states and goals. A 
few authors have also attempted to use plan graphs in proba- 
bilistic planning to compute estimates of the probability that 
propositions can be achieved and actions can be performed. 
This is done by propagating probability information forward 
through the plan graph from the initial conditions through 
each possible action to the action effects, and hence to the 
propositions at the next layer of the plan graph. The prob- 
lem with these calculations is that they make very strong in- 
dependence assumptions - in particular, they usually assume 
that the preconditions for each action are independent of each 
other. This can lead to gross overestimates in probability 
when the plans for those preconditions interfere with each 
other. It can also lead to gross underestimates of probabil- 
ity when there is synergy between the plans for two or more 
preconditions. 

In this paper we introduce a notion of the binary correlation 
between two propositions and actions within a plan graph, 
show how to propagate this information within a plan graph, 
and show how this improves probability estimates for plan- 
ning. This notion of correlation can be thought of as a contin- 
uous generalization of the notion of mutual exclusion (mutex) 
often used in plan graphs. Atone extreme (cofrelaflon^O) 
two propositions or actions are completely mutex. With cor- 
relation = 1, two propositions or actions are independent, and 
with correlation > 1, two propositions or actions are syner- 
gistic. Intermediate values can and do occur indicating differ- 
ent degrees to which propositions and action interfere or are 
synergistic. We compare this approach with another recent 
approach by Bryce that computes probability estimates using 
Monte Carlo simulation of possible worlds in plan graphs. 

Introduction 

Plan graphs are commonly used in planning to help compute 
heuristic “distance” estimates between states and goals. A 
few authors have also attempted to use plan graphs in proba- 
bilistic planning to compute estimates of the probability that 
propositions can be achieved and actions can be performed. 
This information can then be used to help guide a proba- 
bilistic planner towards the most effective, actions for max- 
imizing probability or for achieving the goals with a given 
probability threshold. 

Typically, probability information is given for the propo- 
sitions in the initial state and is propagated forward through 


theplan-graphrinamiannersimilarto the propagationof cost 
and resource estimates in classical planning. The probability 
of being able to perform an action is taken to be the prob- 
ability that its preconditions can be achieved, which is usu- 
ally approximated as the product of the probabilities of the 
preconditions. The probability of a particular action effect 
is taken as the product of the action probability and proba- 
bility of the effect given the action. Finally, the probability 
of achieving a proposition at the next layer is then taken to 
be either the sum or maximum of the probabilities for the 
different effects matching that proposition. As an example, 
consider the plan graph layer shown in Figure 1 where we 
have two actions a and b each with two preconditions and 
two unconditional effects. Suppose that the probabilities for 
the propositions p, q, and r are .8, .5, and .4 as shown in the 
diagram. The probability that action a is possible would then 
be the probabilty of the conjunction p A q which would be 
.8(.5) = .4. Similarly, the probabilty for action b would be 
.5(.4) = .2. Action a produces effect e with certainty (prob- 
abilty 1), so e simply inherits the probabilty of .4 from a. 
Similarly, action b produces effect g with probabilty .5, so 
the probability of g can be calculated as .2(,5) = .1. The 
calculation for the effect f is a bit harder because both ac- 
tions a and b can produce /, and we could in fact develop a 
plan that uses them both to increase the chances of /. Using 
a alone, the probability of / is .4(.5) — .2, and using b alone 
the probability is .2(1) = .2, so the probability of / using 
both actions is .2 4- -2 — .2(.2) — .36. 
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Figure 1 : A plan graph layer with simple probabilty calcu- 
lations made using the independence assumption. 
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The problem with these simple estimates is that they as- 
sume independence between all pairs of propositions and all 
pairs of actions in the plan graph. This is frequently a very 
bad assumption, if two propositions are produced by the 
same action (e.g. e and /), they are not independent of each 
other, and computing the probability of the conjunction by 
taking the product of the individual probabilities can result 
in a significant underestimate. Conversely, if two proposi- 
tions are mutually exclusive, then the probability of achiev- 
ing them both is zero, and the product of their probabilities 
will be a significant overestimate. In our example, we first 
assumed that the propositions p, q and r were independent 
of each other when computing the probabilities of actions 
a and b. Even if this is so, we then proceeded to assume 
th at actions a and b were independent, when computing th e 
probability of effect /. Clearly this is wrong, since a and b 
share a precondition. 

One obvious way to improve the estimation process 
would be to propagate and use mutual exclusion informa- 
tion, and assign a probability of zero to actions with mutex 
preconditions at a given level. However, this only helps with 
the extreme case where propositions or actions are mutex. 
It does not help with cases of synergy, or with cases where 
propositions are not strictly mutex, but it is much “harder” 
(less probable) to achieve them both. 

To attempt to address this problem, we introduce a more 
general notion which we call “correlation” 1 to capture both 
positive and negative interactions between pairs of propo- 
sitions, pairs of actions, and pairs of action effects. In the 
section that follows, we first give a formal definition of our 
notion of correlation. We then show how to compute and 
use correlation information within a plan graph to get bet- 
ter probability estimates. Finally we show some preliminary 
results, and compare this technique with another recent tech- 
nique developed by Bryce, Kambhampati, & Smith (2006b). 

Definitions and Representation 

Action Representation 

Similar to the representation used in (Bryce, Kambhampati 
& Smith 2006b) an action a is taken to have: 

• an enabling precondition, Pre(a) 

• a set of probabilistically weighted outcomes, #i(a) 

The enabling precondition Pre(a) is a conjunction of liter- 
als, just as for an action in probabilistic PDDL (PPDDL) 
(Younes et al 2005; Younes & Littman 2004) or an ordi- 
nary classical action in PDDL (McDermott 1998). Each out- 
come <3>i(a) has a weight Wi(a) giving the probability that 
the outcome is realized, and consists of a conjunction 
of conditional effects <f>ij (a) of the form: 


normal form presented in (Rintanen 2003). 2 


Correlation 


Formally, we define the correlation between two proposi- 
tions, two actions, or two effects x and y as: 


C(x,y) 


__ Pr(x A y) 

' Pr(x) Pr(j/) 


( 1 ) 


which by Bayes Rule can also be seen as: 

= PrQjy) 

Pr(x) 

_ Pr(3/|s) 

Pr(y) 

Correlation is a continuous quantity that can range from zero 
to plus infinity. Essentially, it measures how much more or 
less probable it is that we can establish x and y together as 
opposed to if we could establish them independently. It has 
the following characteristics: 


C(x,y) ~ 0 if x and y are mutex 

== 1 if x and y are independent 

= =? p^y if x and y are completely 

correlated 3 

More generally, 0 < C{x, y) < 1 means that there is some 
interference between the best plans for achieving x and y 
so it is harder (less probable) to achieve them both than 
to achieve them independently. Similarly, 1 < C(x, y) < 
1/ Pr(:r) means that there is some amount of synergy be- 
tween plans for achieving x and y, so it is easier (more 
probable) to achieve them both than to achieve them inde- 
pendently. 

Instead of computing and keeping mutex information in 
the plan graph, we will compute correlation information be- 
tween all pairs of propositions and all pairs of actions at 
each level. It is worthwhile noting that for a pair of propo- 
~sitions“or^efronsnr“andirwe--could-dnstead"ehoose-to-di-“ 
recily store the probability Pr (x Ay), or either of the two 
conditional probabilities Pr(x|y) or 'Pr(y|ic) instead of the 
correlation C(x, y). This is because these quantities are es- 
sentially equivalent - from our definition of correlation and 
Bayes Rule any of these quantities can be computed from 
any other. We have chosen to introduce the notion of corre- 
lation and store this quantity because: 


1. it is symmetric, unlike the conditional values. 

2. we only need to store it for cases where it is not one - i.e. 
the propositions/actions are not independent 

3. it can.be easily interpreted and understood in terms, of the 
intuitive concepts of mutex,. independence, and synergy. 


Pij £ ij 

where both and Sij are conjunctions of literals. Of 
course, may be empty, in which case is an uncondi- 
tional effect This representation of effects follows the 1ND 

1 Not to be confused with the traditional statistical notion of cor- 

relation. 


2 The representation in PPDDL (Younes etal. 2005; Younes & 
Littman 2004) is a bit more general since it allows arbitrary nesting 
of conditional effects and probabilistic outcomes. We have chosen 
to use the 1ND normal form here because it is a bit easier to work 
with, and PPDDL can be expanded into this form. 

3 x cannot occur without y, and vice versa, which means that 
their probabilities must be the same. 
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Computing Probability and Correlation 

To compute probability and correlation information in a plan 
graph, we begin at the initial state (level 0) and propagate 
information forward through the plan graph to subsequent 
levels (just as with construction and propagation in ordinary 
classical plan graphs). In the subsections that follow, we 
give the details of how to do this beginning with the initial 
proposition layer and working forward to actions, then ef- 
fects, and finally to the next proposition layer. 4 


Computing Action Probabilities 


Suppose that we have the probabilities and correlation infor- 
mation for propositions at a given level of the plan graph. 
How do we use -this information to compute probabilities 
andxorrelationinformation-for-the-subsequent-actionTayer? 
First consider an individual action a with preconditions 
{xi, . . . , x n }. The probability that the action can be ex- 
ecuted is the probability that all the preconditions can be 
achieved: 

Pr(a) = Pr(#i A ... A x n ) 

= Pr(xx) Pr(x 2 |a:i) . . . Pr^Xx . . . x„_x) (2) 

If the propositions x* are all independent this is just the 
usual product of the individual probabilities of the precondi- 
tions. However, if they are not independent then we need the 
conditional probabilities, Pr(x*|xi . . . x*_ i). Since we have 
pairwise correlation information we can readily compute the 
first of these terms: 


Pr^lxi) = C{ X 1 ,x 2 )'Pt:{x2) 


However, to compute the higher order terms (i.e. i > 2) we 
must make an approximation. Applying Bayes Rule we get: 


Pr(xi|xi ... . Xi-i) 


Pr(xx A ... A i \xj) Pr(xj) 
Pr(xi A ... A x^ i) 


If we make the assumption that xi . . . x*_i are independent 
-for purposes ofjthis, computation. _we_get:.._„ 


Pr(xi|xi ... Xi-{) 


Pr(a;i |a?j) . . . Prfoj-x fe) Prfa) 
Pr(xi) . . .Pr(a:i_i) 


Applying our analogue of Bayes Rule again i— 1 times, we 
get: 


Pr(a:i|*i....a^_i) 


_ Prfajsx) Pr (sjfo-x) , , 

Pr(a*) Pr(s<) {l) 

= Pr(xi)C(xi, xi) . . . C(xi\xi-i) 


= Pv(xi) J| C(xi,Xj) (3) 

i- 

Returning to the calculation of: 

Pr(a) = Pr(xi A ... A x n ) 

— Pr(xi) Pr(x 2 |xi) . . . Pr(x n |xi . . . x n - 1) 


4 Because we are dealing with actions that have conditional ef- 
fects, we will be distinguishing between effects in a plan graph, 
and the subsequent literal or proposition layer, as is done in 
EPP (Koehler et at 1997) and (Bryce, Kambhampati, & Smith 
2006a; 2006b). 


if we plug in the above expression for the Pr(xi \xi . . . x*_i ) 
we get 

Pr(a) == Pr(xi A ... A x n ) 


= n pr ( s i) n cfav ;) i (4) 

i— l...n 1 

Several properties of this approximation are worth noting: 

1. the above expression is easy to compute and does not de- 
pend on the order of the propositions. 


2. If the Xi are independent, the C{xi,Xj) are 1 and the 
above simplifies to the product of the individual proba- 
bilities. 


3. If any Xi and Xj are mutex then C{x^Xj) — 0 and the 

“ affovffeTpressiffrihecom^ are pos- 

itive but less than one then the probability of the conjunc- 
tion is less than the product of the probabilities of the in- 
dividual elements. 

4. If the C(x iy Xj) are greater than one, there is synergy be- 
tween the conjuncts. The probability of the conjunction 
is greater than the product of the probabilities of the in- 
dividual conjuncts, but less than or equal to the minimum 
of those probabilities. 

While these properties are certainly desirable, and match 
our intuitions, it is reasonable to ask how good the approxi- 
mation in Equation 4 is in other cases. As it turns out, for a 
conjunction with n terms. Equation 4 turns out to be exact if 
only about n of the possible n 2 C(xi , Xj) are not equal to 1 . 
More precisely: 

Theorem 1 Consider the undirected graph consisting of a 
node for each conjunct x^ and an edge between Xi and Xj 
whenever Xi and Xj are not independent (C(xi,Xj) is not 
equal to 1 j. If this graph has no cycles, then Equation 4 is 
exact 

As an example, consider the simple case of: 
~”Pr(ff7\frA”c) -=TPr(^)rP^ 

Our graph consists of the three nodes a, b and e, and zero to 
three edges depending on the C’s. If b and c are independent, 
there are only two edges in the graph, and no cycle, so the 
theorem states that Equation 4 is exact. To see this, with b 
and c independent the above expansion becomes: 

Pr(a Ab Ac) = Pr(a) Pr(6|a) Pr(c|a) 

= Pr (a) Pr (b)C(a, b) Pr (c)C(a, c) 

Which is the approximation in Equation 4, since C(6, c) — 1 
More generally, the proof of this theorem relies on the fact 
that a graph without cycles can be represented as a tree: 


Proof: Suppose we have a conjunction xj A — A x n that 

obeys the conditions of the theorem. Since the graph has no 
cycles, it can be arranged as a tree. Without loss of gener- 
ality, assume the conjuncts are in the same order as a depth 
first traversal of that tree. 

In general, we know that: 

Pr(xi A ... A x n )— Pr(xi |xi . . . x*_ 1) 

i=U 
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But since the conjuncts are ordered according to a depth first 
traversal of the tree, each conjunct Xi has only one predeces- 
sor Xj — x par ty (its parent in the tree) for which C(xi, Xj) 
is not one. As a result,: 

= Pr(Xi\% party) 

— Pv(Xi) O (%i | X p ar ty) 

This means that: 

Pr(zi A ... A Xn) = Vl(Xi)C{Xi, Xpar(i)) 

But since C(xi,Xj) == 1 for all j < i and j ^ par(^) there 
is no harm in adding these terms and we get: 

Pr(a) ~ Pr(xi A — A x n ) 

= n Pr (^t) n c ( x i> x i) 

which is Equation 4. B 


Computing Correlation Between Actions 

As with propositions, the probability' that we can execute 
two actions, a and 6, may be more or less than the product 
of their individual probabilities. If the actions axe mutually 
exclusive (in the classical sense) then the probability that we 
can execute them both is zero. Otherwise, it is the probabil- 
ity that we can establish the union of the preconditions for 
(he two actions. 

Pr(a Ab) — 0 * if a and b are mutex 

.== Pr (/\(Pre(a) U Pre(6))) otherwise 

Using Equation 4 we can compute the probability of the con- 
junction Pr (/\(Pre(a) U Pre(fr))). By our definition of cor- 
relation, Equation 1, we can then compute the correlation 
between two actions a and b. 

As an example, consider the plan graph in Figure 1 again. 

Pr(a A&) — Pr ^ y\ (Pre(a) U Pre(6)) ^) 

■= Pr(p AqAr) 

= Pr(p) Pr(g) Pr (r)C(p, q)C(q, r)C(p,r) 

= .8(.5)(.4) = .16 

assuming that the correlations are all one. The correlation 
between a and b is therefore: 


C(a,b) 


Pr(a A b) .16 
Pr(a) Pr(5) “ .4(.2) 


Computing Effect Probabilities and Correlation 

Given the tools we have developed so far, it is relatively 
straightforward to compute the probability of an individual 
action effect. Let be an outcome of action a with weight 
Wi, and let faj = pij be a conditional effect in 

If the effect is unconditional - that is the antecedent pty is 
empty -then: 

Pr(%) = m Pr(o) . . 

However, if the antecedent p^ is not empty, there is the pos- 
sibility of interaction (positive or negative) between the pre- 
conditions of a and the antecedent p^. As a result, to do the 


computation right we have to compute the probability of the 
conjunction of the preconditions and the antecedent: 

Pr(£ij) = Wi Pr (A(Pre(a) U #,)) 

For convenience, we will refer to the weight Wi associated 
with an effect as u?(%). We will also refer to the union 
of the action preconditions and the antecedent pjj for an ef- 
fect Sij as simply the condition of 'ey and denote itCndfey). 
For an effect e, the above expression then becomes simply: 

Pr(e) = w(e) Pr Cnd(e))^ 

As with actions, we can compute the probability of the con- 
junction of Cnd(e) using the approximation in Equation 4. 

We can also compute the correlation between two differ- 
-ent- effeets-just as- we did with ae-tions-For-two effects,-e and 
/ we have: 

Pr(e A /) = w(e)w{f) Pr (/\(Cnd(e) U Cnd(/))) (5) 

As before, the probability of the conjunction of Cnd(e) U 
Cnd(/) using the approximation in Equation 4. By our def- 
inition of correlation. Equation 1, we can then compute the 
correlation between the two effects e and /. 

As an example, consider the two unconditional effects e 
and g from Figure I . Since both these effects are uncon- 
ditional, Cnd(e) and Cnd(g) are just the preconditions of a 
and b respectively. As a result: 

Pr(e A g) = w(e)w(g)Vv (^/\(Cnd(e) UCnd(^))^ 

= w(e)w(g) Pr(p A q A r) 

= 1(.5)(.8)(.5)(.4) 

= .08 


since p, q and r were assumed to be independent. Using this, 
we get: 


C{e,g) - 


Pr(e A g ) 
Pr(e)_Pr(5)„ 


.08 

AU1. 


Note that Equation 5 for Pr(e A /) applies whether the 
effects e and / are from (he same or different actions. In 
the case where they are effects of the same action, there will 
be overlap of the action preconditions between Cnd(e) and 
Cnd (/). However, the antecedents of the conditional effects 
may be quite different, and there can be interaction (positive 
or negative) between literals in those antecedents, which will 
be captured by the probability calculation in Equation 5. 


Computing Proposition Probabilities 

Computing the probability for a proposition is complicated 
by the fact that there may be many actions with effects that 
produce the proposition, and we are not limited to using only 
one such action or effect For example, if two action effects 
e and / both produce proposition p with probability .5, then 
we may be able to increase our chances of achieving p by 
performing both of them. However, whether or not this is 
a good idea depends upon the correlation between the two 
effects. If the effects are independent or synergistic, then 
it is advantageous. If the two effects are completely mu- 
tex ( C(e , /) ■= 0), then it is not a good idea. If there is 
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some degree of mutual exclusion between the actions (i.e. 
0 < C(e ? f) < 1) then the decision depends on the specific 
probability and correlation numbers. 

Suppose we choose a particular set of effects E = 
{ei, . . . , ek} that produce a particular proposition p. Intu- 
itively, it would seem that the probability that one of these 
effects would yield p is: 

Pr(ei V . . . V e fc ) 

Unfortunately, this isn’t quite right. By choosing a particular 
set of effects to try to achieve p, we are committing to (try- 
ing to) establish the conditions for all of those effects, which 
means establishing both the action preconditions and the an- 
tecedents of each of the conditional effects. There may be 
interaction between those conditions (positive or negative) 
thatlncreases or decreases our“chances“fdr each oflheTf- 
fects. The above expression essentially assumes that all of 
the effects are independent of each other. 

In this case, the correct expression for Pr(p) using a set 
of effects E is both complicated and difficult to compute. 
Essentially we have to consider the probability table of all 
possible assignments to the conditions for the effects E, and 
multiply the probability of each as signment by the probabil- 
ity that the effects enabled by that assignment will produce 
p . Let T(E) be the set of all possible i truth assign- 

ments to the conditions in Cnd(f?). Formally we get; 

Pr ~ L Pr ( r ) Pr (pl r ) ( 6 ) 

r€T(B) 

where Pr(ps) refers to the probability of p given that we are 
using the effects E to achieve p. 

As an example, consider the calculation of the probability 
for the proposition / in Figure 1 assuming that we are using 
both the effects from action a and action 6. The set of condi- 
tions for these (unconditional) effects is just the union of the 
preconditions for a and b which is (p, g, r}. There are eight 
possible truth assignments to this set, but only three of them 
permit at least one of the actions: 

p A q A ->r permits a but not b 
~^p A q Ar permits b but not a 
pAqAr permits both a and b 
The probabilities for these truth assignments are: 

Pr(p Aq A - , r) .== .8(.5)(.6) = .24 
Pr(-np A q A r) = .2(.5)(.4) = .Q4 
Pr(p A q A r) — .8(.5)(.4) = .16 
The probability for g using both actions is therefore: 

Pr (g) == .24(.5) + .04(1) -j- .16(.5 + 1 - .5(1)) = .32 

This calculation was fairly simple because we were only 
dealing with three propositions p, q and r and they were 
independent More generally, however, an expression like 
Pr(p A q A r«r) is problematic when r is not independent of 
the other two propositions, since we do not have correlation 
information for the negated proposition. There are a number 
of approximations that one can use to compute such proba- 
bilities. For our purposes, we assume that two propositions 


are independent if correlation information is not available. 
Thus, in this case we make the assumption that: 

Pr(p Aq A ->r) = Pr(p A q) Pr(->r) 

We now return to the problem of computing the proba- 
bility for a proposition p. In theory we could consider each 
possible subset E' of effects E that match the proposition p 
and compute the maximum: 

max Pr(p£') (7) 

E'CE 

and use Equation 6 to expand and compute Pr(p#/). Un- 
fortunately, when there are many effects that can produce a 
proposition this maximization is likely to be quite expensive, 
because 1) we would need to consider all possible subsets of 
the se t of effects, and _2)_ in Equati on 6 we wo ul d ha ve to 
consider all possible truth assignments to the conditions for 
each set of effects. As a result, some approximation is in 
order. One possibility is a greedy approach that adds effects 
one at a time, as long as they still increase the probability. 
More precisely : 

1. Let E be the set of effects matching p 

2. let Eq be the empty set of effects, let Po = 0 

3. let e be an effect in E not already in E^u and let P* = 

PrOeUSi-J- If 
e maximizes P* 

and 

P* > Pi-i 
then set 

Ei = eUEi-i 

Pi = P* 

Using this procedure the final set Pi will be a lower bound 
on: 

max Pr( pe') 

Even this a p proximation is somewhat expensive to com- 
pute, because it requires repeated computation of Pr(p#/) 
at each stage using equation 6. A different approximation 
that avoids much of this computation is to construct all max- 
imal subsets E l of the effects in E such that there is no pair 
of effects e and / in E f with C(e,f) < 1 (no interference). 
We then compute or estimate Pr(ps' ) for each such subset 
and choose the maximum. This approximation has the ad- 
vantage that we must only calculate Pr (pE f ) for a relatively 
small number of sets. 

Computing Correlation Between Propositions 

Finally, we consider the probability for a pair of proposi- 
tions p and q which will allow us to compute the correlation 
between the propositions. As with a single proposition, this 
calculation is complicated because we want to find the best 
possible set of effects for establishing the conjunction. If we 
let . I? be the set of effects matching proposition p, and F be 
the set of effects matching proposition g, then what we are 
after is: 

Pr(p A q) = max Pr(p£?/ A qpr) 

F'CF 
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In order to compute Pr A qpi- we must again resort to 
considering all possible truth assignments for the union of 
the conditions for E' and F f as we did in Equation 6: 

Pr(p& A q F ')= ^ Pr(r) Pr(p A q\r) (8) 

t€T(S'UF') 

Of course this would also be costly to compute, since it 
involves computing a complex expression for all subsets of 
effects in E and F. To approximate this, we could use either 
the greedy strategy developed in the previous section, or the 
strategy of finding maximal non-interfering effect subsets. 

Given Pr(pAq) and the individual probabilities Pr(p) and 
Pr [q) we can compute C(p, q ) trivially from the definition 
in E quation 1. 


Results 

We have developed a preliminary implementation of the 
technique presented above. Correlation and probability in- 
formation is computed using the above methods. This in- 
formation is then used to guide construction of a relaxed 
plan, which is used to guide the POND heuristic search 
planner (Bryce, Kambhampati, Sc Smith 2006a) in a man- 
ner similar to that described in (Bryce, Kambhampati, Sc 
Smith 2006b). The planner is implemented in C and uses 
several existing technologies. It employs the PPDDL parser 
(Younes Sc Littman 2004) for input, and the 3PP planning 
graph construction code (Koehler et al. 1997). Because the 
implementation and debugging is still not complete, we have 
so far only tested the ideas on the small domains Sandcastle- 
67 and Slippery gripper. Figures 2 and 3 show some early 
results for time, plan length, and node expansion for the 
sandcasde-67 and slippery Gripper domains respectively. 
The plots compare 4 different planners: 

• CPlan (Hyafil 8c Bacchus 2004) 

VMcLu^lri^Bxy^erK^bhampatiG^Smnh“2006b)71he^ 
POND planner using Monte Carlo Simulation on plan 
graphs 

• pr-rp, relaxed plan construction using simple plan graph 
probability information computed using independence as- 
sumptions 

• eoir-rp, relaxed plan construction using probability and 
correlation information. 

The other two entries (pr-rp-mx and corr-rp-mx) represent 
variants that are not fully debugged and should therefore be 
regarded as suspect. 

. Generally, performance of the four methods is similar on 
these simple domains. Plans are somewhat longer for pr- 
rp and coir-rp because the objective for these planners is to 
maximize probability rather than minimize the number of 
actions. There is some indication that corr-rp is showing 
less growth in time and number of node expansions as the 
probability threshold becomes high, but additional experi- 
ments are needed to confirm this and examine this behavior 
more closely. 


Discussion and Conclusions 

We have introduced a continuous generalization of the no- 
tion of mutex, which we call correlation . We showed how 
such a notion could be used to improve the computation of 
probability estimates within a plan graph. Our implementa- 
tion of this technique is still preliminary and it is much too 
early to draw any significant conclusions about the practi- 
cality or efficacy of these computations for problems of any 
size. In addition to finishing our implementation and doing 
more significant testing, there are a number of issues that we 
wish to explore: 

Correlation vs Relaxed Plans The approach of keeping 
correlation information is different from the method of us- 
“mg a reiaxed^M _ ^^ti^te^foMbihtyGn~an“ important 
way: relaxed plans are constructed greedily, so a relaxed 
plan to achieve p A q would normally choose the best way 
to achieve p and the best way to achieve q independently. 
This will not always lead to the best plan for achieving the 
conjunction. Correlation information can be used to guide 
(relaxed) plan selection and would presumably give better 
relaxed plans. This is the approach we have taken in our pre- 
liminary implementation. Of course there is always a trade- 
off between heuristic quality and computation time, and this 
is something we intend to investigate further. 

Admissibility Although probability estimates computed 
using correlation information should be more informative, 
they are not admissible. The primary reason for this is that 
keeping only binary correlation information, and approxi- 
mating the probability of a conjunction using only binary 
correlation information can both underestimate and overesti- 
mate the probability of the conjunction. Note, however, that 
the usual approach of estimating probability by assuming in- 
dependence is also not admissible for the same reason. Sim- 
ilarly, relaxed plans do not provide an admissible heuristic - 
they can underestimate probabilitjT bec"auTe“tfre' relaxed'pran ' 
may not take full advantage of synergy between actions in 
the domain. It is possible to construct an admissible heuris- 
tic for probability by taking: 

• the probability of a conjunction to be the minimum prob- 
ability of the conjuncts, 

• the probability of a proposition as the sum of all the prob- 
abilities of the producing effects. 

However, this heuristic is very weak and not likely to be 
very effective. It is not yet clear whether we can construct a 
stronger admissible heuristic using correlation. 

Correlation in the Initial State The mechanism we have 
described easily admits the use of correlation information 
between propositions in the initial state. That information 
would be treated in the same was as at any other level in 
the plan graph. Thus, if the initial state has Pr (p A q) — 

.5 and Pr (pp A ->q) == .5 we could represent this as 
Pr(p) = Pr (q) .= Pr (-#) — Pr (->q) = .5 and C(p, q) = 
G{pp, nq) = = 2. The limitation of this approach is 
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Figure 2: Run times (s). Plan lengths, and Expanded Nodes vs. probability threshold for sandcastle-67 




Figure 3: Run times (s), Plan lengths , and Expanded Nodes vs. probability threshold for slippery gripper 


that binary correlation can only approximate joint probabil- 
ity information for conjunctions larger than two: 

Bayesian Networks There are a number of similarities be- 
tween techniques we have used here, and methods used in 
Bayesian Networks. We speculate that the calculation of 
probability information for individual actions and pairs of 
actions could be modeled using a simple Bayes net with 
nodes for the preconditions and actions, arcs between the 
preconditions and corresponding actions and arcs between 
~~pMrTofpfecondition~sntet~^ 
equal to one). These later arcs would be labeled with the 
conditional probability corresponding to the correlation. It 
would be necessary to structure the network carefully to 
avoid cycles among the preconditions. The more complex 
calculations for propositions would require influence dia- 
grams with choice nodes for each of the establishing effects. 
There doesn’t seem to be any particular advantage to doing 
this, however. Solution of this influence diagram would re- 
quire investigating all possible sets of the decisions, which 
corresponds to the unwieldy maximization over all subsets 
of establishing effects. 

Cost Computation in Classical Planning The idea that 
we have explored here - a continuous generalized of mutex 
- is not strictly limited to probabilistic planning. A similar 
notion of the “interference” between two proposiitons or two 
actions could be used in classical planning to improve plan 
graph estimates of cost or resource usage. To do this we 
could define “interference” as: 

I(x } y) = Cost(x A y) — ( Cost(x ) + Cost(y)) 


— Cost(y\x) Cost(y) 

= Cost(x\y) — Cost(x) 

Positive interference means that there is conflict between 
two propositions, actions or effects, and that it is more ex- 
pensive to achieve the conjunction than to achieve them sep- 
arately. Interference of plus infinity corresponds to mutex. 
Negative interference corresponds to synergy between the 
propositions, meaning that achieving them together is easier 
than achieving them independently. Interference of zero cor- 
responds to independence. Essentially, this can be seen as 
'the logarithm of lhe~definitibh foFcoirelatio^ given in Equa- 
tion 1. 

The computation of interference for actions, effects and 
propositions is very similar to what we have described 
above. The primary difference is that computations for 
propositions are significantly simpler because there is no 
need to maximize over all subsets of possible effects that 
give rise to a proposition. Although we have worked out the 
equations and propagation rules for this notion of interfer- 
ence, we have not yet implemented or tested this idea. We 
intend to investigate this in the near future. 
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